Receiver Operating Characteristics (ROC) curves have long been used to evaluate classifier performance in many fields (e.g. signal detection and machine learning). An ROC curve provides information on the tradeoff between the hit rate (true positives) and the false alarm rates (false positives). In order to draw the ROC curve both positive and negative examples are needed.
In some applications, for example, machine condition monitoring, there are plenty of negative examples. However the positive examples are either rare, or do not fully describe the overall set of the possible positive examples. Instead of the positive examples, some rules about the positive examples are known. For example, if a sensor drifts off from the set of observed states by a certain amount, a fault has occurred.
In the prior art, ROC curves have not been considered as a criteria to compare different machine condition monitoring models. Some prior art references have used qualitative criteria to compare different methods and monitoring models. Still other prior art references have used accuracy. Some prior art references have used true and false positive rates, which correspond to a single point on the ROC curve, while others have used indications of increase/decrease in the signals as a model goodness criterion. Nevertheless, to date, no prior art references have taught or suggested the use of ROC curves as a means for comparing different operating models, especially those operating models used for monitoring power plants.
Accordingly, what is needed is a method of evaluating different models used for monitoring system. What is also needed is a method of selecting a model for monitoring a power plant. What is also needed is a method of evaluating monitoring models that may be used for other systems.